FIG. 1a shows prior art cache circuitry 100 and FIG. 1b shows a timing diagram for the prior art cache circuitry. As observed in FIG. 1a, the prior art cache circuitry 100 includes a tag array 101 and a data array 102. The tag array 101 keeps “tags” (identifiers) of the cache lines that are kept in the data array 102. Under a common sequence of events, such as a “demand read”, both the tag array 101 and the data array 102 are accessed as part of the same transaction. For example, in the case of a demand read, first the tag array 101 is accessed to see if the desired cache line is in the data array 102. If so (cache hit), the data array 102 is accessed to fetch the desired cache line.
According to a traditional design point, the data storage capacity of the data array 102 is desired to be large which corresponds to the use of smaller, but slower, data storage cells within the data array 102. As a consequence of the use of slower cells, two cycles are needed to access the data array 102. Moreover, the tag array 101 is traditionally implemented with the same type of storage cells as the data array 102. Hence, a complete sequence of first accessing the tag array 101 and then accessing the data array 102 (e.g., in the case of a cache hit) requires four cycles (2 cycles for the tag array access plus 2 more cycles for the data array access).
Because two cycles are needed to access the tag array 101, cache access requests can not be serviced on immediately consecutive cycles. That is, as observed in FIG. 1b, an immediately following cache access request 121 can not be serviced on a cycle that immediately follows a preceding cache access request 120. Rather, an entire cycle 123 must be “skipped” after the first request 120 before the second request 121 can be serviced.
Note that the prior art caching circuitry 100 also includes a main request queue 103 for queuing cache access requests. According to one approach, requests from the main queue 103 are fed into a FIFO 104 that directly supplies cache access requests to the tag array 102. According to one embodiment of this approach, requests are serviced in strict order according to their arrival to the main queue 103. Here, servicing logic 105 services requests from the FIFO 104 consistent with the timing discussed above with respect to FIG. 1b. That is, servicing logic 104 is designed to wait a full cycle after servicing a request from the FIFO 104 before servicing a next request from the FIFO 104.
In other approaches (not shown), the different sources of incoming cache access requests are serviced on a round robin basis. For example, a semiconductor chip that has four cores and two external links may have individual queues for each of the six sources (four cores and two links), and, the queues are fairly serviced on a round robin basis. Strict ordering of request services based on arrival to the cache circuitry need not be strictly adhered to according to this approach. Here again, because of the two cycle tag access requirement, even if round robin servicing is utilized, the servicing logic nevertheless will not service a next request until after a full cycle has elapsed since the serving of the most recently serviced request.